{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise - Cross-Validation and the Train-Test Split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this exercise, you will apply what you have learned about splitting the dataset into training and test sets, impute missing values and scale features the correct way to avoid data leakage, and perform k-fold cross-validation in order to get more reliable and representative estimates of the model's performance on unseen data.  \n",
    "\n",
    "The dataset is a modified version of the [\"Housing Prices Dataset\" from Kaggle](https://www.kaggle.com/datasets/yasserh/housing-prices-dataset)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# DO NOT MODIFY - imports\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Data Preparation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Other than a few missing values which were introduced intentionally for the purpose of this demo, the dataset is clean and free from duplicated rows and other issues. You do not need to write your own code in this section. However, please read this section and inspect the code thoroughly to understand how the dataset is being set up for the next step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>price</th>\n",
       "      <th>area</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>stories</th>\n",
       "      <th>mainroad</th>\n",
       "      <th>guestroom</th>\n",
       "      <th>basement</th>\n",
       "      <th>hotwaterheating</th>\n",
       "      <th>airconditioning</th>\n",
       "      <th>parking</th>\n",
       "      <th>prefarea</th>\n",
       "      <th>furnishingstatus</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>13300000</td>\n",
       "      <td>7420.0</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>2</td>\n",
       "      <td>yes</td>\n",
       "      <td>furnished</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>12250000</td>\n",
       "      <td>8960.0</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>3</td>\n",
       "      <td>no</td>\n",
       "      <td>furnished</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>12250000</td>\n",
       "      <td>9960.0</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>2</td>\n",
       "      <td>yes</td>\n",
       "      <td>semi-furnished</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>12215000</td>\n",
       "      <td>7500.0</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>3</td>\n",
       "      <td>yes</td>\n",
       "      <td>furnished</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>11410000</td>\n",
       "      <td>7420.0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>yes</td>\n",
       "      <td>yes</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>2</td>\n",
       "      <td>no</td>\n",
       "      <td>furnished</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      price    area  bedrooms  bathrooms  stories mainroad guestroom basement  \\\n",
       "0  13300000  7420.0         4          2        3      yes        no       no   \n",
       "1  12250000  8960.0         4          4        4      yes        no       no   \n",
       "2  12250000  9960.0         3          2        2      yes        no      yes   \n",
       "3  12215000  7500.0         4          2        2      yes        no      yes   \n",
       "4  11410000  7420.0         4          1        2      yes       yes      yes   \n",
       "\n",
       "  hotwaterheating airconditioning  parking prefarea furnishingstatus  \n",
       "0              no             yes        2      yes        furnished  \n",
       "1              no             yes        3       no        furnished  \n",
       "2              no              no        2      yes   semi-furnished  \n",
       "3              no             yes        3      yes        furnished  \n",
       "4              no             yes        2       no        furnished  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# DO NOT MODIFY - Data loading and inspection\n",
    "df = pd.read_csv(\"Housing_Modified_2.csv\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "price                0\n",
       "area                13\n",
       "bedrooms             0\n",
       "bathrooms            0\n",
       "stories              0\n",
       "mainroad             0\n",
       "guestroom            0\n",
       "basement             0\n",
       "hotwaterheating      0\n",
       "airconditioning      0\n",
       "parking              0\n",
       "prefarea             0\n",
       "furnishingstatus     0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# DO NOT MODIFY - Check for missing values\n",
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will impute the missing values in the `area` column.  \n",
    "But first, run the cell below to convert the categorical \"`yes`/`no`\" columns to ones and zeros (integers)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>price</th>\n",
       "      <th>area</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>stories</th>\n",
       "      <th>mainroad</th>\n",
       "      <th>guestroom</th>\n",
       "      <th>basement</th>\n",
       "      <th>hotwaterheating</th>\n",
       "      <th>airconditioning</th>\n",
       "      <th>parking</th>\n",
       "      <th>prefarea</th>\n",
       "      <th>furnishingstatus</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>13300000</td>\n",
       "      <td>7420.0</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>furnished</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>12250000</td>\n",
       "      <td>8960.0</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>furnished</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>12250000</td>\n",
       "      <td>9960.0</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>semi-furnished</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>12215000</td>\n",
       "      <td>7500.0</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>furnished</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>11410000</td>\n",
       "      <td>7420.0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>furnished</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      price    area  bedrooms  bathrooms  stories  mainroad  guestroom  \\\n",
       "0  13300000  7420.0         4          2        3         1          0   \n",
       "1  12250000  8960.0         4          4        4         1          0   \n",
       "2  12250000  9960.0         3          2        2         1          0   \n",
       "3  12215000  7500.0         4          2        2         1          0   \n",
       "4  11410000  7420.0         4          1        2         1          1   \n",
       "\n",
       "   basement  hotwaterheating  airconditioning  parking  prefarea  \\\n",
       "0         0                0                1        2         1   \n",
       "1         0                0                1        3         0   \n",
       "2         1                0                0        2         1   \n",
       "3         1                0                1        3         1   \n",
       "4         1                0                1        2         0   \n",
       "\n",
       "  furnishingstatus  \n",
       "0        furnished  \n",
       "1        furnished  \n",
       "2   semi-furnished  \n",
       "3        furnished  \n",
       "4        furnished  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# DO NOT MODIFY - Data preparation\n",
    "# Convert \"yes\" and \"no\" to 1 and 0\n",
    "yes_no_columns = [\"mainroad\", \"guestroom\", \"basement\", \"hotwaterheating\", \"airconditioning\", \"prefarea\"]\n",
    "df[yes_no_columns] = df[yes_no_columns].map({\"yes\": 1, \"no\": 0}.get)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "price                 int64\n",
       "area                float64\n",
       "bedrooms              int64\n",
       "bathrooms             int64\n",
       "stories               int64\n",
       "mainroad              int64\n",
       "guestroom             int64\n",
       "basement              int64\n",
       "hotwaterheating       int64\n",
       "airconditioning       int64\n",
       "parking               int64\n",
       "prefarea              int64\n",
       "furnishingstatus     object\n",
       "dtype: object"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the cell below to one-hot-encode the `furnishingstatus` column with the first resulting column (`furnished`) dropped to avoid multicollinearity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "price                                int64\n",
       "area                               float64\n",
       "bedrooms                             int64\n",
       "bathrooms                            int64\n",
       "stories                              int64\n",
       "mainroad                             int64\n",
       "guestroom                            int64\n",
       "basement                             int64\n",
       "hotwaterheating                      int64\n",
       "airconditioning                      int64\n",
       "parking                              int64\n",
       "prefarea                             int64\n",
       "furnishingstatus_semi-furnished       bool\n",
       "furnishingstatus_unfurnished          bool\n",
       "dtype: object"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# DO NOT MODIFY - One-hot encoding `furnishingstatus`\n",
    "df = pd.get_dummies(df, columns=[\"furnishingstatus\"], drop_first=True)\n",
    "df.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are now ready to split the data, impute missing values and scale the features if need be."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Train-Test Split and Proper Imputation and Scaling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create the feature set, the matrix `X`, consisting of all columns but `price`. Then create the target, the array `y`, comprised of the values in the `price` column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# FILL IN - Create feature set `X` and target `y`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Split the data into training and testing sets using a 70/30 split. Shuffle the data while you split it, using a random seed of 52."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# DO NOT MODIFY - imports\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# FILL IN - Split the data into training and testing sets (70% train, 30% test) with a random state of 52\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Impute missing values in the `area` column using the `SimpleImputer` class from Scikit-Learn. Use the `median` strategy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# DO NOT MODIFY - imports\n",
    "from sklearn.impute import SimpleImputer\n",
    "\n",
    "# FILL IN - Fit the imputer on the training data, then transform the training AND test data using the fitted imputer\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below, we pick out columns of data that were originally numeric (and not just 0 or 1). Scale these features using a MinMaxScaler the correct way. - **HINT:** Only pass `X_train[numeric_columns]` and `X_test[numeric_columns]`, not all columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "#  DO NOT MODIFY - Features that were originally numeric (and not just 0 or 1)\n",
    "numeric_columns = [\"area\", \"bedrooms\", \"bathrooms\", \"stories\", \"parking\"]\n",
    "\n",
    "# DO NOT MODIFY - imports\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "\n",
    "# FILL IN - Fit the MinMaxScaler on the training data, then transform the training AND test data using the fitted scaler\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using `describe()`, verify that all values in both sets are between zero and one now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# FILL IN - `describe()` the training set\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# FILL IN - `describe()` the test set\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. K-Fold Cross-Validation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train alinear regression model on the training set and output its *training* score (which, by default, is the R-squared for regression tasks). - **HINT:** Use the `score()` method of the fitted model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# DO NOT MODIFY - imports\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "# FILL IN - Train a linear regression model and output its R-squared score on the training set\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Can we expect a similarly high score on unseen data? Before looking at the holdout (test) set, cross-validate the model using 5-fold CV and output the average score."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# DO NOT MODIFY - imports\n",
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "# FILL IN - Cross-validate the model using 5-fold CV and output the mean R-squared score\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, evaluate the trained model on the test set and output the test score (R-squared). Is it closer to the training score or the average CV score?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# DO NOT MODIFY - imports\n",
    "from sklearn.metrics import r2_score\n",
    "\n",
    "# FILL IN - Evaluate the model on the test set and output its R-squared score\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
